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BACKGROUND OF THE INVENTION 
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Field of the Invention: 



The present invention relates to robots and, also, to 
apparatus for composing video source material. 

In the production of audiovisual films, videos, 
television commercials, movies, and the like, a video camera is 
used to record video images and/or audio signals on a video tape. 
10 The video tape may be carried directly by the camera, as in a video 
camcorder, or the video images from a video camera may be output to 
a video tape recorder (VTR) which records the video images on a 
video tape. The video tape can then be played back to review the 
recorded images . 

15 As frequently occurs during the production of audiovisual 

materials, it is oftentimes necessary to re-shoot some portion of 
the recorded sequence of images. When this occurs, the video tape 
must be rewound to the desired position or frame to enable new 
video images to be recorded over old images or new video images are 

20 recorded on a separate tape. After the new and old images are 
recorded on different tapes, the two video tap<|s are then edited by 
merging the desired portions of the two tapes into a single tape to 
form the desired sequence of video images. A "Aimber of video tape 



editing devices are available to perform such editing operations. 

25 Time code generators are also well known and are used to 

generate standardized time codes. Such time codes are developed 
according to standards set by the SMPTE (Society of Motion Picture 
and Television Engineers) and the EBU (European Broadcasting 
Union) . The time codes are formed of a binary code corresponding 

3 0 to hours, minutes, seconds and frame number of each frame of a 
video signal starting from a zero point and continuing during the 
entire video image running time. The time code information is 



encoded onto a recording medium by a video tape recorder on a 
separate track from the tracks containing the video information for 
each frame of the video signal. The recorded time code information 
acts as an address to identify each frame to permit electronic 
5 editing of video tapes. 

During editing, time code readers are employed to locate 
a particular frame on a video tape. However, such editing is a 
time consuming process requiring a considerable amount of skill and 
costly time code generators, time code readers and tape 

10 synchronizers which result in high post production costs to form a 
complete audiovisual tape. 

As described above, it is often necessary during the 
production of audiovisual material to re-shoot the same sequence of 
images many times before a final sequence is obtained. The camera 

15 and/or the object (s) being filmed must be moved through the same 
path of movement with only the desired variations being introduced 
in each separate re-shoot. Various devices have been employed to 
repeatedly move an object and/ or camera through a predetermined 
path. 

20 Robots have been developed, primarily for industrial 

applications, to move a tool, etc., mounted at the end of an end 
effector on a robot arm in a predetermined I multi-axis path of 
movement according to a stored program executed by a robot 
controller. A 

25 It is known to mount a camera on the movable arm of a 

robot so as to be able to repeatedly move the camera through a 

predetermined path of movement while the camera is operated to 

record a sequence of video images. However, heretofore, there has 

jJ 

been no known attempt to synchronize the movement of the robot as 
30 it executes its stored control program with the advance or reverse 
movement of a video recording medium storing video images in a 
predetermined sequence to enable the robot and the recording medium 
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to be moved in synchronization to a predetermined position 
corresponding to a particular frame on the video tape with the 
robot being positioned in the same position as it was while the 
image was shot. The different speeds of the film transport in the 
5 video camera and/or video tape recorder and the motors controlling 
the position and speed of movement of the robot arm have made such 
synchronization difficult and have resulted in hit or miss efforts 
to position a robot in a particular position in multiple axes 
corresponding to a particular robot position in a particular frame 

10 on a video tape. 

Thus, it would be desirable to provide an apparatus for 
synchronizing robot or machine movement with a video recording 
medium such that the robot or machine can be positioned in the same 
position as recorded on each frame of the video tape. It would 

15 also be desirable to provide an apparatus for use with a robot or 
machine and a video recording apparatus which enables the robot arm 
to be moved to any position corresponding to a particular frame of 
a recorded video image while remaining synchronized in time with 
the video recording. 

20. I SUMMARY OF THE INVENTION 

/"' The present invention is a video tirie code synchronized 

robot control apparatus which synchronizes the position of a robot 
or other automated machine having a camera m sunted on a movable 
robot arm with the frame-by-frame images on] a video recording 

25 medium. 

The apparatus includes a robot or machine having an arm 
movable through a path of movement. A video camera is mounted on 
the arm and generates video signals during movement of the camera 
along the path of movement of the arm. A time code generator means 
3 0 generates time code information for use with the video signals from 
the video camera. A video tape recorder in response to the video 
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signals from the video camera and the time code information from 
the time code generator means records a composite video signal 
formed of the video signal from the video camera and the time code 
information from the time code generator onto a video recording 
5 medium. 

A time code reader means is responsive to the composite 
video signal output from the video tape recorder and decodes the 
time code information. The decoded time code information is input 
to a robot controller means which controls the path of movement of 

10 the robot arm in accordance with a taught and/or stored program. 
The robot controller, in response to the decoded time code 
information from the time code reader, synchronizes the movement of 
the robot arm along its programmed path of movement with the time 
code information on a video image frame-by-frame basis. 

15 m one embodiment, the video signals from the video 

camera are input to a time code generator which generates time code 
information for each frame of video signals. The video signals and 
the time code information are then output to a video tape recorder 
which records a composite signal of the video signal and the time 

20 code information onto a video tape recording medium. A video 
monitor may also be provided and controlled by the video tape 
recorder for displaying the video images. On a real time basis or 
during playback of the video tape in the vid^S tape recorder, the 
time code information on the video tape is decoded by a time code 

25 reader which outputs the decoded time code information to the robot 
controller. In another embodiment, the output of the video camera 
and the time code information from the time code generator are 
separately input to a video tape recorder which records the 
composite signal formed of the video images'j and the time code 

30 information onto separate tracks of a video recording medium or 
tape. The output from the video tape recorder is decoded by the 
time code reader and input to the robot controller as in the first 
embodiment. 



In yet another embodiment, the video signals from the 
video camera are input to a combined time code generator/reader. 
The time code generator portion of the combined time code 
generator/reader generates time code information and outputs the 
5 video images and the generated time codes to a video tape recorder 
which stores the video images and the time code information on a 
video tape. The output of the video tape recorder is input to the 
combined time code generator/reader which decodes the time code 
information and outputs the decoded time code information to the 
10 robot controller. 

The decoded time code information from the time code 
reader is synchronized with the multi-axis positional coordinates 
of the robot control program by any of a number of different means. 
In an exemplary embodiment, the robot controller, upon receiving 
15 the decoded time code information, generates and stores in memory 
the time code information for each frame of the video signal and 
the multi-axis robot arm positional coordinates corresponding to 
each frame. In this manner, upon receipt of a particular time code 
defining a particular frame of the video signal, the robot 
20 controller can position the robot arm in the positional coordinates 
corresponding to those at which the particular video image was 
originally taken. 

The unigue video time code synchr^ lized robot control 
apparatus of the present invention simplified the production of 
25-,- audiovisual materials by enabling the video images stored frame-by- 
frame on a video tape to be coordinated in timelwith the positional 
coordinates of a multi-axis robot controller This enables the 
robot controller to move its arm carrying thei.video camera to the 
same position in which a particular frame oi| video signals was 
generated. This simplifies the editing of video tapes by enabling 
the robot to be moved to any desired position in synchronization 
with the video images and new video images generated and recorded 
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onto the same tape. This also simplifies post production editing 
of multiple video tapes and insures that the robot, the robot arm 
and the video camera carried by the robot arm are in the same 
identical position corresponding to each frame of the video 
signals. 

BRIEF DESCRIPTION OF THE DRAWING 

The various features, advantages and other uses of the 
present invention will become more apparent by referring to the 
following detailed description and drawing in which: 

Figure 1 is a perspective view of a robot apparatus 
designed to move a video camera through a predetermined, programmed 

path of movement; 

Figure^ 2 is a perspective view of another embodiment of 
a robot apparatus usable with the present invention; 

Figure .. 3 is a block diagram of one embodiment of the 
apparatus of the present invention; 

Figure x 4 is a block diagram of another embodiment of the 
apparatus of the present invention; and 

Figure s 5 is a block diagram of yet another embodiment of 
the apparatus of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Referring now to Figure 1, therd is depicted one 
embodiment of a robot control apparatus usable with the present 
invention. It will be understood that any type of robot, such as 
a gantry-type robot 10 shown in Figure 1, as well as pedestal 
robots may also be used. Further, the term "]|obot" is also meant 
to include any machine which is capable of repeatedly moving an arm 
or member thereof along a predetermined path of movement. 



By way of example only, the robot 10 is an AEG 8000___ 
Series gantry-type robot. This robot 10 includes four frame 
members, only two of which 12 and 14 are shown in Figure 1. The 
frame members 12 and 14 are supported above a floor by a plurality 
of upright legs 16. 

The frame members 12 and 14 are disposed in parallel and 
spaced apart. The frame members 12 and 14 are formed as channel-, 
like members which slidably support horizontal sliders 18 and 20, 
respectively. The horizontal sliders 18 and 20 are slidably 
disposed in the frame members 12 and 14, respectively, and are 
movable along an axis, hereafter denoted as the X axis. A 
horizontal slider drive means is provided for driving the 
horizontal sliders 18 and 20 in a bi-directional manner along the 
frame members 12 and 14. By way of example only, the drive means 
comprises an electric motor 22 mounted at one end of the frame 
member 14. The output shaft of the motor 22 is mounted in a 
bearing and engages a drive pulley, not shown. A similar drive 
pulley is mounted in the frame member 12 and is connected to the 
output shaft of the motor by a coupling shaft 24. A timing belt, 
not shown, is mounted in each of the frame members 12 and 14 and is 
operably coupled to the drive pulleys associated with the motor 22. 
Rotation of the output shaft and drive pulleys by the motor 22 
results in movement of the timing belt which ^frives the horizontal 
sliders 18 and 20 reciprocatingly along the firame members 12 and 

A pair of spaced, channel-like cross members 28 and 30 
are mounted at opposite ends to the horizontal sliders 18 and 20 
and span the horizontal sliders 18 and 20. The cross members 28 
and 30 extend along a Y axis perpendicular to the X axis defined by 
the frame members 12 and 14. A carriage 32 is slidably mounted on 
the cross members 28 and 30 and is reciprocatingly driven along the 
cross members 28 and 30 by a Y axis drive motor 34 mounted on one 
end of the cross members 28 and 30. A timing belt extends through 



the cross member 28 and is reciprocatingly driven through a pulley 
by the Y axis drive motor 34. The timing belt is coupled to the 
carriage 32 and drives the carriage 32 reciprocatingly along the V 
axis upon activation of the Y axis drive motor 34. 

A third or Z axis drive motor 40 is mounted on the 
carriage 32 and reciprocatingly moves a Z axis channel 42 
substantially vertically through the carriage 32. 

An end effector 50 is slidably mounted on the Z axis 
channel 42 and is vertically movable along with the channel 42 in 
response to bi-directional rotation of the output shaft of the Z 
axis motor 40. A conventional video camera 52 is mounted on the 
end effector and is moved by the robot apparatus 10 along a 
programmed multi-axis path of movement. 

The electrical control signals to the various drive 
motors 22, 34 and 40 as well as feedback signals, etc., pass 
through a junction box 54 and are supplied by electrical 
conductors, cables, etc., mounted in cable carriers 26 and 36 to a 
robot controller means 56. Any conventional robot controller 56 
may be employed, such as a Modicon 3240/3220 Flexible Automation 
Controller, for example only. Such a robot controller includes a 
central processing unit or computer which executes a control 
program stored in a memory. A keyboard and a. display, not shown, 
are also provided with the robot cont^aLler 56. As is 
conventional, such robot controllers 56 are adapted for learning a 
particular path of movement of the end effector 50. In such a 
learning or teach mode, the end effector 50 is manually moved or 
jogged through a predetermined multi-axis path of movement and each 
step or sequence of steps are input to the robot controller which 
stores the multi-axis positional coordinates'^ of the robot arm, 
i.e., the Z axis channel 42, at each step. The sequence of steps 
thus defines the predetermined path of movement of the end effector 
50 and the camera 52 mounted on the arm or Z axis channel 42. 
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Figure 2 depicts another embodiment of a robot 60 usable 
with the present invention. The robot 60 includes the same gantry- 
type robot described above and shown in Figure 1. However, in this 
embodiment the Z axis channel 42 is replaced by a second robot 62. 
The second robot 62 is mounted to the Y axis carriage 32 and thus 
is movable by the gantry robot 10 along the X and Z axes under the 
control of the robot controller 56. 

The second robot 62 may be any type of pedestal-type 
robot, such as a Motoman K10 robot sold by Yaskawa Electric Mfg. 
Co., Ltd. The robot 62 includes a movable arm 64 to which a video 
camera 52 is mounted by means of a bracket 66. The robot 62 and 
the movable arm 64, which are controlled by the same robot 
controller 56 controlling the gantry robot 10, provides a greater 
freedom of movement for the camera 52 in conjunction with the X and 
Y axis movements provided by the gantry robot. In both of the 
robots 10 and 62, a rotatable end effector may be mounted on the 
robot arm to support the video camera 52 and provide an additional 
axis of movement to the video camera 52 . 

Referring now to Figure 3, there is depicted one 
20 embodiment of a video time code synchronization control apparatus 
which coordinates video image time codes with the positional 
coordinates stored in the robot controller 56 to coordinate the 
multi-axis position of the robot 56 with the ^ .deo images recorded 
by the video camera 52. As shown in Figure ■ ,, the video signals 
25 output from the video camera 52 during movement of the camera 52 

.t 

are input to a conventional time code generator means 70, such as 
•A a time code generator Model No. CDI-716A sold by Cipher Digital. 
As is well known, the time code generator means 70 generates a 
binary time code for each frame of video sigrikls from the camera 
30 52. The time code signal is in a standardized format as 
established by SMPTE (Society of Motion Picture and Television 
Engineers) or the EBU (European Broadcasting Union) . The binary 
time code acts as an address for each frame of the video image or 



signals and specifies in encoded form the hour, minutes, seconds 
and frame number of running time of each frame of the sequence of 
video images from a zero or start frame. In the SMPTE time code 
format, a new frame of video signals is generated every l/30th of 
5 a second. In the EBU format, each video signal frame is generated 
D every l/24th of a second. Other formats, such as 24 frame per 
second motion picture film formats, are also possible with the 
present invention. 

In the embodiment shown in Figure 3, the video signals 

10 from the video camera 52 and the time code generated by the time 
code generator means 70 are separately output from the time code 
generator means 70 to a conventional video tape recorder (VTR) 72. 
The video tape recorder 72 may be any suitable type of video tape 
recorder, such as a broadcasting-type video tape recorder. Such a 

15 video tape recorder 72 records a composite signal on a suitable 
video image recording medium, such as a video tape, which is formed 
of the video images and the associated time code on a frame-by- O 
frame basis on separate tracks. 

A monitor 74 may optionally be connected to the video 

2 0 tape recorder 72 to display the video images from the camera 52 in 
real time or the video images recorded on the video tape during 
playback. 

A time code reader means 76 is co^ lected to the video 
tape recorder 72 and receives time code information from the video 

25 tape recorder 72 on a frame-by-frame basis. Tpe time code reader 

'it 

means 76, as is conventional, decodes the time code information and 
outputs, according to the present invention, the decoded time code 
information to the robot controller 56. The output signals from 

the time code reader means 76, denoted generally by reference 

'if 

,30 number 78, may be in RS232 serial or RS422 parallel format. 

Further, such output signals from the time code reader means 76 may 
be in the specified form of a serial digital interface standard for 
communicating video equipment with digital equipment. 

10 



An alternate embodiment of the present invention is shown 
in Figure 4 in which the same components as shown in Figure 3 are 
connected in a different configuration, but provide the same 
result. As shown in Figure 4, the video signals from the video 
camera 52 and the time code information from the time code 
generator means 70 are separately input to the video tape recorder 
72. The video tape recorder 72 forms a composite signal of the 
video images and the time code information on a frame-by-frame 
basis and records the video signals and the time code information 
on separate tracks on a video tape recording medium in a normal 
manner. The recorded time codes are then decoded on a frame-by- 
frame basis by the time code reader means 76 and output to the 
robot controller 56. 

Another embodiment is shown in Figure 5 in which the 
video signals from the video camera 52 are input to a combined time 
code generator/reader 80. The time code generator/reader 80 may 
be, by way of example only, a time code reader /generator event 
controller, Model No. CDI-750, sold by Cipher Digital. In the time 
code generator portion of the time code generator/reader 80, time 
code information is generated for each frame of video signals 
received from the video camera 52. The video signals and the time 
code information are output to the video tape recorder 72 on a 
frame-by-frame basis. The video tape recover 72 records both 
signals on a video tape recording medium. Timel code information on 
a frame-by-frame basis is output from the videoftape recorder 72 to 
the reader portion of the combined time code generator/reader 80 
the time code information and outputs the decoded time code signals 
to the robot controller 56. 

An alternate embodiment of the apparatus shown in Figure 
5 may also be provided. In the alternate embodiment, as shown in 
Figure 4, the outputs of the video camera 52 and time code 
information from the combined time code generator/reader 80 may be 
separate input to the video tape recorder 72 to form the composite 
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signal which is recorded on the video tape by the video tape 
recorder 72. 

In a preferred sequence of operations, it will be assumed 
that the robot 10 or 60 has been moved through a teach mode to 
5 learn a desired multi-axis path of movement, which path of movement 
is stored by positional coordinates in the memory of the robot 
controller 56. With the robots 10 or 10 and 60 positioned at their 
respective start positions, execution of the control program by the 
robot controller 56 will cause the robots 10 or 10 and 60 to move 

10 the robot arm through its predetermined path of movement thereby 
moving the video camera 52 along the same predetermined path of 
movement. Activation of the video camera 52 during such movement 
will cause a series of video signals on a frame-by-frame basis to 
be generated and output from the video camera 52. 

15 Using the apparatus shown in Figure 3, the video signals 

from the video camera 56 will be input to the time code generator 
means 70 which will generate time code information for each frame 
of video signal. The video signals and the time code information 
from the time code generator 70 are output to the video tape 

20 recorder 72 which records a composite signal containing the video 
signals and the time code information on a suitable video tape 
recording medium on a frame-by-frame basis. . 

In real time, while the video cameVji 52 is being moved 
through its predetermined path of movement or during playback, the 

25 time code information recorded by the video tap'fe recorder 72 on the 

'i 

video tape is output to the time code reader means 7 6 which decodes 
the time code information and supplies the decoded time code 
information to the robot controller 56. 

According to the present invention, ijhe robot controller 
30 56 via a suitable software control program, circuitry or via an 
external unit communicating with the robot controller 56, 
coordinates the positional coordinates of the robot control program 
with each frame of video signal as defined by the time code 

12 
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information received from the time code reader means 76. This may 
be implemented, by example only, by the robot controller 56 which 
divides each step or path of movement between two points into a 
number of increments, each at a time period corresponding to the 
5 time interval of each frame of video images or signals, i.e., every 
0^ D l/30th or l/24th second. The robot controller 56 Aearte and stores 
in memory the multi-axis positional coordinates of the robot arm at 

i > each -4A-3^feh or l/24th of a second. This links the multi-axis 

f\ 

positional coordinates of the robot with the time code defining 

10 each frame of video signal. 

After the robot controller 56 has completed the control 
program and moved the robot 10 or 60 through its predetermined path 
of movement, the synchronizing information synchronizing the time 
code with the positional coordinates of the robot corresponding to 

15 each frame of video signal may then be utilized to move the robot 
controller to any desired position in its predetermined path of 
movement during advance or rewind of the video tape on which the 
video signals have been recorded by the video tape recorder 72. As 
each frame of video signal on the video tape has a specific time 

20 code associated therewith, the robot controller 56 via the learned 
positional coordinates in memory can determine the multi-axis 
coordinates of the robot 10 or 60 corresponding to each time code 
of video information and thereby move the rc4t 10 or 60 to such 
coordinates as each frame of video information Advances through the 

25 video tape recorder 72. The video tape recorder 72 may be stopped 
at any frame at which time the robot 10 or 60 will be precisely 
positioned in the same positional coordinates as it was when the 
video image recorded on the particular frame on the video tape was 
first taken. This simplifies editing since i^ny portion of the 

3 0 video image on the video tape may be re-shot starting with the 
position of the robot at any position along its predetermined path 
of movement. 
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In summary, there has been disclosed a video time code 
synchronized robot control apparatus which coordinates time code 
information associated with each frame of a video image recorded on 
a video tape with the stored positional coordinates of a robot 
apparatus to enable the robot apparatus to be moved to a position 
corresponding to the position of its components when a particular 
frame of video image was first taken. Editing of the video tape as 
required to re-shoot a particular portion of an overall image 
sequence may easily be done on the same tape by moving the robot to 
any predetermined position along its path of movement corresponding 
to a particular frame of video image. The succeeding portion of 
the robot movement and/or the object being imaged may then be 
varied and new video images generated as the robot moves through 
its existing path of movement or a revised path of movement. 
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